ENG-19808 mlxfwreset before reboot in SR-IOV operator#8
Merged
punkerpunker merged 2 commits intoclark/config-daemon-ib-unbind-fixfrom Feb 20, 2025
Merged
Conversation
|
Thanks for your PR,
To skip the vendors CIs, Maintainers can use one of:
|
clarkzinzow
approved these changes
Feb 20, 2025
punkerpunker
merged commit 1fb8405
into
clark/config-daemon-ib-unbind-fix
15 of 21 checks passed
clarkzinzow
added a commit
that referenced
this pull request
Aug 28, 2025
Add platform build arg. Comment out Mellanox plugin's draining + rebooting for totalVfs + SRIOV_EN configs, which is buggy. Scan GUIDs (#7) * Squash commits into one * Squash commits into one * Cherry-picked types (build fix) * merge issues fix * read GUID from sysfs * node -> port * args mismatch :( * NAD config fix to include guid * port -> node back * rollback partially * rollback partially * rollback partially * rollback partially * rollback partially * rollback partially * rollback partially * fix * bring back pKey to netAttDef definition * pkey proper location * removed excessive log lines * quotes fix ENG-21048 - KernelArgIommuOn instead of KernelArgIommuPt (to enable ATS & ACS) (#9) KernelArgIommuOn instead of KernelArgIommuPt ENG-19808 mlxfwreset before reboot in SR-IOV operator (#8) * bring reboots back * bring reboots back GUIDSavedInUFM config parameter added camelcase -> snake case
clarkzinzow
added a commit
that referenced
this pull request
Aug 28, 2025
Upgrade golangci-lint to work with Go 1.23 Add platform build arg. Comment out Mellanox plugin's draining + rebooting for totalVfs + SRIOV_EN configs, which is buggy. Scan GUIDs (#7) * Squash commits into one * Squash commits into one * Cherry-picked types (build fix) * merge issues fix * read GUID from sysfs * node -> port * args mismatch :( * NAD config fix to include guid * port -> node back * rollback partially * rollback partially * rollback partially * rollback partially * rollback partially * rollback partially * rollback partially * fix * bring back pKey to netAttDef definition * pkey proper location * removed excessive log lines * quotes fix ENG-21048 - KernelArgIommuOn instead of KernelArgIommuPt (to enable ATS & ACS) (#9) KernelArgIommuOn instead of KernelArgIommuPt ENG-19808 mlxfwreset before reboot in SR-IOV operator (#8) * bring reboots back * bring reboots back GUIDSavedInUFM config parameter added camelcase -> snake case
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Well, overall it seems like something wrong on the firmware level enabling the SR-IOV capability, from the kernel perspective things looks okay to me
I see in the sriov-network-operator there's an option to enable mstfwreset after mlxconfig change! I guess that's something that we can try leveraging:
https://github.com/openshift/sriov-network-operator/blob/79cb3c6ae721220754189300539a38c63e38e66c/pkg/plugins/mellanox/mellanox_plugin.go#L215
I think this is going to resolve the reboot loop we're getting when running config-daemon, and I'll try doing it tomorrow.
All paths so far goes to firmware, I don't think I'll find anything more in the kernel, tbh
Infra PR to enable featureGate - https://github.com/togethercomputer/infra/pull/4044